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THE REPLY FILED 03 November 2004 FAILS TO PLACE THIS APPLICATION IN CONDITION FOR ALLOWANCE. 
Therefore, further action by the applicant is required to avoid abandonment of this application. A proper reply to a 
final rejection under 37 CFR 1.113 may only be either: (1) a timely filed amendment which places the application in 
condition for allowance; (2) a timely filed Notice of Appeal (with appeal fee); or (3) a timely filed Request for Continued 
Examination (RCE) in compliance with 37 CFR 1.114. 

PERIOD FOR REPLY [check either a) or b)] 

a) ^ The period for reply expires 3_months from the mailing date of the final rejection. 

b) n The period for reply expires on: (1 ) the mailing date of this Advisory Action, or (2) the date set forth in the final rejection, whichever is later. In 

no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of the final rejection. 

ONLY CHECK THIS BOX WHEN THE FIRST REPLY WAS FILED WITHIN TWO MONTHS OF THE FINAL REJECTION. See MPEP 

706.07(f). 

Extensions of time may be obtained under 37 CFR 1.136(a). The date on which the petition under 37 CFR 1.136(a) and the appropriate extension 
fee have been filed is the date for purposes of determining the period of extension and the corresponding amount of the fee. The appropriate extension 
fee under 37 CFR 1.17(a) is calculated from: (1) the expiration date of the shortened statutory period for reply originally set in the final Office action; or 
(2) as set forth in (b) above, if checked. Any reply received by the Office later than three months after the mailing date of the final rejection, even if 
timely filed, may reduce any earned patent term adjustment. See 37 CFR 1.704(b). 

1 A Notice of Appeal was filed on . Appellanfs Brief must be filed within the period set forth in 

37 CFR 1.192(a), or any extension thereof (37 CFR 1.191(d)), to avoid dismissal of the appeal. 

2. n The proposed amendment(s) will not be entered because: 

(a) □ they raise new issues that would require further consideration and/or search (see NOTE below); 

(b) □ they raise the issue of new matter (see Note below); 

(c) □ they are not deemed to place the application in better form for appeal by materially reducing or simplifying the 

issues for appeal; and/or 

(d) □ they present additional claims without canceling a corresponding number of finally rejected claims. 

NOTE: . 

3. n Applicant's reply has overcome the following rejection(s): . 

4. n Newly proposed or amended claim(s) would be allowable if submitted in a separate, timely filed amendment 

canceling the non-allowable claim(s). 

5.13 The a)n affidavit, b)n exhibit, or c)^ request for reconsideration has been considered but does NOT place the 
application in condition for allowance because: See Attachment -A . 

6. n The affidavit or exhibit will NOT be considered because it is not directed SOLELY to issues which were newly 

raised by the Examiner in the final rejection. 

7. n For purposes of Appeal, the proposed amendment(s) a)n will not be entered or b)n will be entered and an 

explanation of how the new or amended claims would be rejected is provided below or appended. 

The status of the claim(s) is (or will be) as follows: 

Claim(s) allowed: . 

Claim(s) objected to: . 

Claim(s) rejected: 1-36 . 

Claim(s) withdrawn from consideration: . 

8. n The drawing correction filed on is a)n approved or b)n disapproved by the Examiner. 

9. n Note the attached Information Disclosure Statement(s)( PTO-1449) Paper No(s). . 

10. n Other: 
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ATTACHMENT - A: ADVISORY ACTION 

1 . This communication is in response to the Applicants' response dated November 
23, 2004. Applicant's arguments filed on November 23, 2004 have been fully 
considered. Applicant's arguments, filed on November 23, 2004 under 35 U.S.C. 103 
(a) are not persuasive. 

Arguments 

2.1 As per the applicant's argument that "the prior art simply formulates a discrete MDP in 
terms of linear programming, which is well known; the applicant's invention on the other hand, 
is a more general method that works in a continuous space, continuous action setting; the 
appHcant's invention attempts to approximate the correct value function, with which acting 
optimally in each state requires solving a Linear Programming (LP) problem that incorporates 
this value function; the prior art does not teach or suggest these aspects of the applicant's 
invention", the Examiner respectfully disagrees. 

Viniotis teaches continuous space and continuous action setting (Page 654, CLl, Para 2). 
It teaches approximating the correct value function and solving the Linear programming problem 
that incorporates the value function (Page 654, CLl, Para 3 to CL2, Para 5). 

2.2 As per the applicant's argument that "the Office Action asserts that Viniotis teaches a 
state space for the MDP is a polyhedron in a Euclidean space; the Office Action imputes more 
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into Viniotis than it actually teaches; in Viniotis, A is a constant matrix, not a state space; 
nowhere does Viniotis refer to a state space for the MDP as a polyhedron in a Euclidean space", 
the Examiner respectfully disagrees. 

Viniotis states that the solution to the Linear Programining problem is an extreme point 
(Page 654, CL4, Para 6); extreme points form a polyhedron (Page 654, CL4, Para 6). One of 
ordinary skill in the art would have known that such polyhedron existed in the Euclidean space (a 
multi-dimensional space). The constraints of the linear program are lines in the multi- 
dimensional space forming the edges of the polyhedron. The constraints are defined by the 
states. Therefore, the state space of the linear program exists in an Euclidean space and is defined 
by a polyhedron. It is well known that a Markov decision Problem (MDP) is equivalent to a 
Linear Program; a MDP problem can be generally formulated as an equivalent Linear Program 
(Page 652, CLl, Para 4). Therefore, one of ordinary skill in the art would conclude that a state 
space for the MDP is a polyhedron in a Euchdean space. 

2.3 As per the applicant's argument that "the Office Action asserts that Viniotis teaches one 
or more actions that are feasible in a state of the state space are linearly constrained with respect 
to the state; . . . Viniotis does not teach or suggest that actions that are feasible in a state of the 
state space are linearly constrained with respect to state, in the context of where a state space for 
the MDP is a polyhedron in Euclidean space; . . . Viniotis merely states that the state is a linear 
function of the control action Zk", the Examiner respectfully disagrees. 



Application/Control Number: 09/607,5 1 3 Page 4 

Art Unit: 2123 

Viniotis states that the state is a linear function of the control actions (Page 652, CL2, 
Para 8). One of ordinary skill in the art knows that if x is a linear function of y, then y is a linear 
function of x. Therefore, it is clear that actions are linear functions of state. Selecting an optimal 
policy (set of actions) reduces .to minimizing a linear functional; this minimization is constrained, 
since the states generated by the policy have to belong to the state space, a subset of nonnegative 
integers (Page 653, CLl, Para 1). Therefore, it is obvious that the actions are constrained by the 
state, where the state space is in the Euclidean space. 

2.4 As per the applicant's argument that "the Office Action asserts that Viniotis teaches 
building a value function for the state using representations that facilitate the computation of 
approximately optimal actions at any given state by linear programming; Viniotis does not teach 
that building a value function for the state using representations and facilitating the computation 
of approximately optimal actions at any given state by linear programming; Viniotis teaches only 
the formulation of an MDP and the definition of a value function, as well as that the state is 
linear function of the control actions", the Examiner respectfully disagrees. 

Viniotis teaches building a value function for the state using representations that facilitate 
the computation of approximately optimal actions at any given state by linear programming 
(Page 653, CLl, Para 9 to Page 654, CLl, Para 4; Page 652, CL2, Para 6 and Para 8). 

2.5 As per the applicant's argument that "the Office Action asserts that Schneider teaches 
building one or more approximations from above and from below to a value function for the state 
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using representations that facilitate computation of approximately optimal actions at any given 
state by linear programming; Schneider does not teach or suggest building one or more 
approximations from above and from below to a value ftinction for the state using 
representations that facilitate computation of approximately optimal actions at any given state by 
linear programming; . . . Schneider merely describes how the solution to the MDP is an 
approximate value function, specific to the current demand forecasts, which can be used to 
generate optimal scheduling decisions online", the Examiner respectfully disagrees. 

Schneider teaches that the solution to the MDP is a value function and a method for 
generating an approximate value of this function (Page 2722, CLl, Para 2). Schneider also 
teaches that the solution to an MDP is an approximate value function (Page 2724, CL2, Para 6). 
Schneider teaches that the value function can be represented as a function of states and actions 
(Page 2725, CLl, Para 1). Trajectories through the MDP model are generated repeatedly using 
the current approximation of the value function (Page 2725, CL2, Para 4). For noisy versions, 
one could use noisy outcomes directly from the stochastic simulation (Page 2726, CLl, Para 3). 
Schneider teaches building one or more approximations from above and from below to a value 
function for the state using representations (Page 2722, CLl, Para 2; Page 2724, CL2, Para 6), as 
value function approximation is an effective technique for both deterministic and noisy scenarios 
(Page 2722, CLl, Para 2); and approximation allows solving large scale MDPs (Page 2722, CL2, 
Para 2). 
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3. In view of the above explanation, the request for reconsideration has been 
considered but is not persuasive and does not place the application in condition for 
allowance. 

K. Thangavelu 
Art Unit 2123 
December 22, 2004 




